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(57) In a picture feature extraction device, picture 
data having been determined as a moving picture by a 
picture data identifying section is subjected to being 
taken out the data corresponding to the entire frames or 
a part of the frames by a feature data extracting frame 
selecting means, upon which a resized picture generat- 
ing means generates a resized picture from the data of 
each frame, and a frequency analyzing means analyzes 
a frequency of the resized picture having been gener- 
ated. Then a DC component/partial AC component 
extracting means takes out DC components and a part 
of AC components, after which a feature extraction 
means generates feature data of the whole frames hav- 
ing been selected. The picture data identified as still pic- 
ture data is subjected to having a resized picture 
generated by the resized picture generating means from 
the data of each frame, after which the frequency ana- 
lyzing means executes a frequency analysis on the 
resized picture having being generated. Then as the DC 
component/partial AC component extracting means 
takes out DC components and a part of AC components 
obtained as a result of the analysis in order to generate 
a feature data. 
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Description 

BACKGROUND OF THE INVENTION 

[0001] The present invention relates to a picture 
feature extraction device, picture retrieving device, as 
well as a method of picture feature extraction, and a 
method of retrieving pictures. 

Description of the Related Art 

[0002] As to the conventional picture feature extrac- 
tion device, picture retrieving device and methods 
thereof, it is typical that they are used in retrieving simi- 
lar pictures or sections of moving pictures from the pic- 
tures being stored. 

[0003] Still picture has been often described using 
color feature. Both a color histogram and dominant 
colors are typically used as picture feature data for 
retrieval. Layout is also a useful feature and can be rep- 
resented using the combination of grid structure, which 
divides a still picture into blocks and uses a color histo- 
gram and local dominant colors for each block. One 
example of a prior art picture retrieving method using a 
reference picture is disclosed in Japanese Patent Laid- 
Open Publication No. 8-249349. In accordance with the 
disclosed retrieving method, the method divides a still 
picture into blocks and uses dominant colors of each 
block as feature data. However, as they need a relatively 
large number of bits for their representation, the similar- 
ity evaluation process requires a large amount of calcu- 
lation. Therefore, this kind of combination is not very 
suitable especially for video segment retrieval, which 
requests many repetition of similarity calculation. One 
example of a prior art video segment retrieval method is 
disclosed in United States Patent 5805746. The dis- 
closed retrieving method reduces the calculation cost 
using the combination of grid structure and local aver- 
age colors for high speed detection of only long and 
identical video segment, like TV commercials. As this 
description represents the color layout roughly, this 
method cannot retrieve still picture and short sections 
with sufficient accuracy. 

[0004] The following problems are recognized in the 
above-mentioned prior art methods. 
[0005] First, in the conventional methods, the fea- 
ture data becomes large, and therefore, it requires a 
large-scale storage unit for storing the feature data. 
Second, a speed required in retrieving decreases 
because the feature data is large. Third, since the 
matching is conducted between the entire inputted mov- 
ing picture feature data and each point of time of the 
moving picture data within the data base subjected to 
retrieval, the time to be consumed for the matching 
process becomes enormous. Particularly, it is a notable 
problem that the data base becomes larger in scale. 
Fourth, if the feature data represents the color layout of 
pictures roughly, still pictures and short sections cannot 



be retrieved with sufficient accuracy. 
SUMMARY OF THE INVENTION 

5 [0006] It is therefore an object of the present inven- 
tion to provide a picture feature extraction device, pic- 
ture retrieving device, and methods thereof for 
generating picture feature data and retrieving pictures, 
respectively, capable of retrieving similar still and mov- 

10 ing pictures from still and moving picture data precisely 
and at high speed, so as to generate still picture feature 
data and moving picture feature data requiring less 
memory capacity. 

[0007] In accordance with a first aspect of the 

is present invention, there is provided a picture feature 
extraction device comprising: a resized picture generat- 
ing means generating a resized picture from still picture 
data; a frequency analyzing means executing a fre- 
quency analysis on said resized picture having been 

20 generated by said resized picture generating means; 
and a DC/partial AC component extracting means tak- 
ing out DC components and a part of AC components 
obtained as a result of the analysis by said frequency 
analyzing means as a picture feature data. 

25 [0008] In accordance with a second aspect of the 
present invention, there is provided a picture feature 
extraction device comprising: a still picture data base 
including a still picture data storing means storing said 
still picture data, and a still picture feature data storing 

30 means storing said still picture feature data generated 
for each still picture data stored in said still picture data 
storing means. 

[0009] In accordance with a third aspect of the 
present invention, there is provided a picture feature 

35 extraction device comprising: a feature data extracting 
frame selecting means taking out some data corre- 
sponding to the entire frames or a portion of the frames 
from the moving picture data, selecting the data sub- 
jected to processing; a resized picture generating 

40 means generating a resized picture from data corre- 
sponding to each frame having been taken out; a fre- 
quency analyzing means executing a frequency 
analysis on said resized picture having been generated 
by said resized picture generating means; a DC/partial 

45 AC component extracting means taking out DC compo- 
nents and a part of AC components obtained as a result 
of the analysis by said frequency analyzing means as a 
frame feature data; and a moving picture feature data 
producing means producing a moving picture feature 

so data by collecting said frame feature data outputted 
from said DC/partial AC component extracting means. 
[0010] In accordance with a fourth aspect of the 
present invention, there is provided a picture feature 
extraction device comprising: a moving picture data 

55 base including a moving picture data storing means 
storing said moving picture data, and a moving picture 
feature data storing means storing said moving picture 
feature data generated for each moving picture data 
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stored in said moving picture data storing means. 
[0011] In accordance with a fifth aspect of the 
present invention, there is provided a picture feature 
extraction device comprising: a picture data identifying 
means identifying whether the inputted picture data is of 
still pictures or of moving pictures; a still picture feature 
extraction means including a resized picture generating 
means generating a resized picture from the picture 
data identified as still picture data by said picture data 
identifying means, a frequency analyzing means exe- 
cuting a frequency analysis on said resized picture hav- 
ing been generated by said resized picture generating 
means, and a DC/partial AC component extracting 
means taking out DC components and a part of AC 
components obtained as a result of the analysis by said 
frequency analyzing means as a still picture feature 
data; and a moving picture feature extraction means 
including a feature data extracting frame selecting 
means taking out some data corresponding to the entire 
frames or a portion of the frames from the picture data 
identified as moving picture data by said picture data 
identifying means, selecting the data subjected to 
processing, a resized picture generating means gener- 
ating a resized picture from data corresponding to each 
frame having been taken out, a frequency analyzing 
means executing a frequency analysis on said resized 
picture having been generated, a DC/partial AC compo- 
nent extracting means taking out DC components and a 
part of AC components obtained as a result of the anal- 
ysis by said frequency analyzing means as a frame fea- 
ture data, and a moving picture feature data producing 
means generating a moving picture feature data by col- 
lecting said frame feature data outputted from said 
DC/partial AC component extracting means. 
[0012] In accordance with a sixth aspect of the 
present invention, there is provided a picture feature 
extraction device comprising: a picture data base 
including a picture data storing means storing said pic- 
ture data, and a picture feature data storing means 
using each picture data stored in said picture data stor- 
ing means for storing the still picture feature data gener- 
ated by said still picture feature extraction means and 
the moving picture feature data generated by said mov- 
ing picture feature extraction means as a picture feature 
data. 

[0013] In accordance with a seventh aspect of the 
present invention, there is provided a picture feature 
extraction device wherein said resized picture generat- 
ing means divides a still picture into blocks, and takes 
out color data representative of each block from each 
block. 

[0014] In accordance with an eighth aspect of the 
present invention, there is provided a picture feature 
extraction device wherein in case when said picture 
data is compressed by blocks, said resized picture gen- 
erating means takes out DC components from each 
block. 

[0015] In accordance with a ninth aspect of the 



present invention, there is provided a picture retrieving 
device comprising: a still picture generating means 
including a resized picture generating means generat- 
ing a resized picture from still picture data, a frequency 

s analyzing means executing a frequency analysis on 
said resized picture having been generated by said 
resized picture generating means, and a DC/partial AC 
component extracting means taking out DC compo- 
nents and a part of AC components obtained as a result 

10 of the analysis by said frequency analyzing means as a 
picture feature data, a still picture data base including a 
still picture feature data storing means storing the still 
picture feature data previously being generated by said 
still picture feature extraction means on the basis of the 

is still picture data, and a still picture data storing means 
storing said still picture data; and a still picture feature 
data correlation measuring means calculating a correla- 
tion value between the still picture feature data gener- 
ated by said still picture feature extraction means on the 

20 basis of the inputted still picture data and the still picture 
feature data stored in said still picture data base, the still 
picture feature data correlation measuring means 
retrieving the still picture within the still picture data 
base having a high correlation value as a candidate of 

25 similar still picture. 

[0016] In accordance with a tenth aspect of the 
present invention, there is provided a picture retrieving 
device comprising: a moving picture feature extraction 
means including a feature data extracting frame select- 

30 ing means taking out some data corresponding to the 
entire frames or a portion of the frames from the moving 
picture data, selecting the data subjected to processing, 
a resized picture generating means generating a 
resized picture from data corresponding to each frame 

35 having been taken out, a frequency analyzing means 
executing a frequency analysis on said resized picture 
having been generated by said resized picture generat- 
ing means, a DC/partial AC component extracting 
means taking out DC components and a part of AC 

40 components obtained as a result of the analysis by said 
frequency analyzing means as a frame feature data, 
and a moving picture feature data producing means 
producing a moving picture feature data by collecting 
said frame feature data outputted from said DC/partial 

45 AC component extracting means; a moving picture data 
base including a moving picture feature data storing 
means storing the moving picture feature data previ- 
ously being generated by said moving picture feature 
extraction means on the basis of the moving picture 

so data, and a moving picture data storing means storing 
said moving picture data; and a moving picture feature 
data correlation measuring means calculating a correla- 
tion value between the moving picture feature data gen- 
erated by said moving picture feature extraction means 

55 on the basis of the inputted moving picture data and the 
moving picture feature data stored in said moving pic- 
ture data base, the moving picture feature data correla- 
tion measuring means retrieving the moving picture 
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segment within the moving picture data base having a 
high correlation value as a candidate of similar moving 
picture segment. 

[001 7] In accordance with an eleventh aspect of the 
present invention, there is provided a picture retrieving 
device comprising: a picture data identifying means 
identifying whether the inputted picture data is of still 
pictures or of moving pictures; a still picture feature 
extraction means including a resized picture generating 
means generating a resized picture from the picture 
data identified as still picture data by said picture data 
identifying means, a frequency analyzing means exe- 
cuting a frequency analysis on said resized picture hav- 
ing been generated by said resized picture generating 
means, and a DC/partial AC component extracting 
means taking out DC components and a part of AC 
components obtained as a result of the analysis by said 
frequency analyzing means as a still picture feature 
data; a moving picture feature extraction means includ- 
ing a feature data extracting frame selecting means tak- 
ing out some data corresponding to the entire frames or 
a portion of the frames from the picture data identified 
as moving picture data by said picture data identifying 
means, selecting the data subjected to processing, a 
resized picture generating means generating a resized 
picture from data corresponding to each frame having 
been taken out. a frequency analyzing means executing 
a frequency analysis on said resized picture having 
been generated by said resized picture generating 
means, a DC/partial AC component extracting means 
taking out DC components and a part of AC compo- 
nents obtained as a result of the analysis by said fre- 
quency analyzing means as a frame feature data, and a 
moving picture feature data producing means generat- 
ing a moving picture feature data by collecting said 
frame feature data outputted from said DC/partial AC 
component extracting means; a picture data base 
including a picture feature data storing means storing 
the still picture feature data previously being generated 
by said still picture feature extraction means on the 
basis of the still picture data and the moving picture fea- 
ture data previously being generated by said moving 
picture feature extraction means on the basis of the 
moving picture data, and a picture data storing means 
storing said still picture data and said moving picture 
data; a still picture feature data correlation measuring 
means calculating a correlation value between the still 
picture feature data generated by said still picture fea- 
ture extraction means on the basis of the inputted still 
picture data and the picture feature data stored in said 
picture data base; and a moving picture feature data 
correlation measuring means calculating a correlation 
value between the moving picture feature data gener- 
ated by said moving picture feature extraction means on 
the basis of the inputted moving picture data and the 
picture feature data stored in said picture data base, 
said still picture feature data correlation measuring 
means retrieving the still picture within the picture data 



base having a high correlation value as a candidate of 
similar still picture, said moving picture feature data cor- 
relation measuring means retrieving the moving picture 
segment within the picture data base having a high cor- 
5 relation value as a candidate of similar moving picture 
segment. 

[0018] In accordance with a twelfth aspect of the 
present invention, there is provided a picture retrieving 
device wherein said moving picture feature data correla- 
te tion measuring means divides the moving picture fea- 
ture data having been generated by said moving picture 
feature extraction means on the basis of said inputted 
moving picture data, and calculates in turn a partial cor- 
relation value between a part of the moving picture fea- 
rs ture data of the moving picture included in said moving 
picture data base and a moving picture feature data of 
each division of the divided moving picture feature data, 
upon which eliminating the moving picture segment 
included in said moving picture data base from the can- 
20 didates of similar moving pictures when it has a feature 
data with a partial correlation value lower than a prede- 
termined threshold value. 

[001 9] In accordance with a thirteenth aspect of the 
present invention, there is provided a picture retrieving 

25 device wherein said moving picture feature data correla- 
tion measuring means divides the moving picture fea- 
ture data having been generated by said moving picture 
feature extraction means in respect with said inputted 
moving picture data, and calculates in turn a partial cor- 

30 relation value between a part of the moving picture fea- 
ture data of the moving picture included in said moving 
picture data base and a partial moving picture feature 
data of each division of the divided moving picture fea- 
ture data, upon which eliminating the moving picture 

35 segment included in said moving picture data base from 
the candidates of similar moving pictures at the time 
when it is acknowledged that a number of partial moving 
picture feature data having a feature data with a partial 
correlation value lower than a predetermined threshold 

40 value surpasses a predetermined threshold value. 

[0020] In accordance with a fourteenth aspect of 
the present invention, there is provided a picture retriev- 
ing device wherein said resized picture generating 
means divides a picture into blocks, and takes out from 

45 each block color information representative of each 
block. 

[0021] In accordance with a fifteenth aspect of the 
present invention, there is provided a picture retrieving 
device wherein said resized picture generating means 

so takes out DC components from said each block when 
said picture data is compressed by blocks. 
[0022] In accordance with a sixteenth aspect of the 
present invention, there is provided a picture feature 
extraction method comprising: a resized picture gener- 

55 ating step for generating a resized picture from still pic- 
ture data; a frequency analyzing step for executing a 
frequency analysis on said resized picture having been 
generated by said resized picture generating step; and 
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a DC/partial AC component extracting step for taking 
out DC components and a part of AC components 
obtained as a result of the analysis at said frequency 
analyzing step as a picture feature data. 
[0023] In accordance with a seventeenth aspect of s 
the present invention, there is provided a picture feature 
extraction method comprising: a still picture data storing 
step for storing said still picture data; and a still picture 
feature data storing step for storing said still picture fea- 
ture data generated for each still picture data stored at 10 
said still picture data storing step. 
[0024] In accordance with an eighteenth aspect of 
the present invention, there is provided a picture feature 
extraction method comprising: a feature data extracting 
frame selecting step for taking out some data corre- 15 
sponding to the entire frames or a portion of the frames 
from the moving picture data, selecting the data sub- 
jected to processing; a resized picture generating step 
for generating a resized picture from data correspond- 
ing to each frame having been taken out; a frequency 20 
analyzing step for executing a frequency analysis on 
said resized picture having been generated at said 
resized picture generating step; a DC/partial AC compo- 
nent extracting step for taking out DC components and 
a part of AC components obtained as a result of the 25 
analysis at said frequency analyzing step as a frame 
feature data; and a moving picture feature data produc- 
ing step for producing a moving picture feature data by 
collecting said frame feature. data outputted from said 
DC/partial AC component extracting step. 30 
[0025] In accordance with a nineteenth aspect of 
the present invention, there is provided a picture feature 
extraction method comprising: a moving picture data 
storing step for storing said moving picture data; and a 
moving picture feature data storing step for storing said 35 
moving picture feature data generated for each moving 
picture data stored at said moving picture data storing 
step. 

[0026] In accordance with a twentieth aspect of the 
present invention, there is provided a picture feature 40 
extraction method comprising: a picture data identifying 
step for identifying whether the inputted picture data is 
of still pictures or of moving pictures; a still picture fea- 
ture extraction step including a resized picture generat- 
ing step for generating a resized picture from the picture 45 
data identified as still picture data at said picture data 
identifying step, a frequency analyzing step for execut- 
ing a frequency analysis on said resized picture having 
been generated at said resized picture generating step, 
and a DC/partial AC component extracting step for tak- so 
ing out DC components and a part of AC components 
obtained as a result of the analysis at said frequency 
analyzing step as a still picture feature data; and a mov- 
ing picture feature extraction step including a feature 
data extracting frame selecting step for taking out some 55 
data corresponding to the entire frames or a portion of 
the frames from the picture data identified as moving 
picture data at said picture data identifying step, select- 



ing the data subjected to processing, a resized picture 
generating step for generating a resized picture from 
data correspond ng to each frame having been taken 
out, a frequency analyzing step for executing a fre- 
quency analysis on said resized picture having been 
generated at said resized picture generating step, a 
DC/partial AC component extracting step for taking out 
DC components and a part of AC components obtained 
as a result of the analysis at said frequency analyzing 
step as a frame feature data, and a moving picture fea- 
ture data producing step for generating a moving picture 
feature data by collecting said frame feature data out- 
putted at said DC/partial AC component extracting step. 
[0027] In accordance with a twenty-first aspect of 
the present invention, there is provided a picture feature 
extraction method comprising: a picture data storing 
step for storing said picture data; and a picture feature 
data storing step using each picture data stored at said 
picture data storing step for storing the still picture fea- 
ture data generated at said still picture feature extrac- 
tion step and the moving picture feature data generated 
at said moving picture feature extraction step as a pic- 
ture feature data. 

[0028] In accordance with a twenty-second aspect 
of the present invention, there is provided a picture fea- 
ture extraction method wherein said resized picture 
generating step divides a still picture into blocks, and 
takes out from each block color data representative of 
each block. 

[0029] In accordance with a twenty-third aspect of 
the present invention, there is provided a picture feature 
extraction method wherein in case when said picture 
data is compressed by blocks, said resized picture gen- 
erating step takes out from each block DC components. 
[0030] In accordance with a twenty-fourth aspect of 
the present invention, there is provided a picture retriev- 
ing method comprising: a still picture feature extraction 
step including a resized picture generating step for gen- 
erating a resized picture from still picture data, a fre- 
quency analyzing step for executing a frequency 
analysis on said resized picture having been generated 
at said resized picture generating step, and a DC/partial 
AC component extracting step for taking out DC compo- 
nents and a part of AC components obtained as a result 
of the analysis at said frequency analyzing step as a 
picture feature data; a still picture feature data correla- 
tion measuring step for calculating a correlation value 
between the still picture feature data of the still picture 
within a still picture data base where the still picture fea- 
ture data of the still picture data having been previously 
generated at said still picture feature extraction step is 
stored together with the still picture data and the still pic- 
ture feature data generated at said still picture feature 
extraction step on the basis of the inputted still picture 
data; and a similar still picture retrieving step for retriev- 
ing the still picture within said still picture data base hav- 
ing a high correlation value with respect to the still 
picture feature data generated on the basis of the input- 
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ted still picture data as a candidate of similar still picture. 
[0031] In accordance with a twenty-fifth aspect of 
the present invention, there is provided a picture retriev- 
ing method comprising: a moving picture feature extrac- 
tion step including a feature data extracting frame 
selecting step for taking out some data corresponding to 
the entire frames or a portion of the frames from the 
moving picture data, selecting the data subjected to 
processing, a resized picture generating step for gener- 
ating a resized picture from data corresponding to each 
frame having been taken out, a frequency analyzing 
step for executing a frequency analysis on said resized 
picture having been generated by said resized picture 
generating step, a DC/partial AC component extracting 
step for taking out DC components and a part of AC 
components obtained as a result of the analysis at said 
frequency analyzing step as a frame feature data, and a 
moving picture feature data producing step for produc- 
ing a moving picture feature data by collecting said 
frame feature data outputted at said DC/partial AC com- 
ponent extracting step; a moving picture feature data 
correlation measuring step for calculating a correlation 
value between the moving picture feature data of the 
moving picture within a moving picture data base where 
the moving picture feature data of the moving picture 
data having been previously generated at said moving 
picture feature extraction step is stored together with the 
moving picture data and the moving picture feature data 
generated at said moving picture feature extraction step 
on the basis of the inputted moving picture data; and a 
similar moving picture retrieving step for retrieving the 
moving picture within said moving picture data base 
having a high correlation value with respect to the mov- 
ing picture feature data generated on the basis of the 
inputted moving picture data as a candidate of similar 
moving picture. 

[0032] In accordance with a twenty-sixth aspect of 
the present invention, there is provided a picture retriev- 
ing method wherein said moving picture feature data 
correlation measuring step has a step of dividing the 
moving picture feature data having been generated by 
said moving picture feature extraction step on the basis 
of said inputted moving picture data, and a step of cal- 
culating in turn a partial correlation value between a 
part of the moving picture feature data of the moving 
picture included in said moving picture data base and a 
moving picture feature data of each division of the 
divided moving picture feature data, and said similar 
picture retrieving step eliminates the moving picture 
segment included in said moving picture data base from 
the candidates of similar moving pictures when it has a 
feature data with a partial correlation value lower than a 
predetermined threshold value. 
[0033] In accordance with a twenty-seventh aspect 
of the present invention, there is provided a picture 
retrieving method wherein said moving picture feature 
data correlation measuring step has a step of dividing 
the moving picture feature data having been generated 



by said moving picture feature extraction step on the 
basis of said inputted moving picture data, and a step of 
calculating in turn a partial correlation value between a 
part of the moving picture feature data of the moving 

5 picture included in said moving picture data base and a 
partial moving picture feature data of each division of 
the divided moving picture feature data, and said similar 
picture retrieving step eliminates the moving picture 
segment included in said moving picture data base from 

w the candidates of similar moving pictures at the time 
when it is acknowledged that a number of partial moving 
picture feature data having a feature data with a partial 
correlation value lower than a predetermined threshold 
value surpasses a predetermined threshold value. 

15 [0034] In accordance with a twenty- eighth aspect of 
the present invention, there is provided a picture retriev- 
ing method comprising: a first picture feature data corre- 
lation measuring step for calculating a correlation value 
between a still picture feature data of a still picture in 

20 inputted still picture data generated by a still picture fea- 
ture extraction step including a resized picture generat- 
ing step for generating a resized picture from the 
inputted picture data identified as still picture data, a fre- 
quency analyzing step for executing a frequency analy- 

25 sis on said resized picture having been generated at 
said resized picture generating step, and a DC/partial 
AC component extracting step for taking out DC compo- 
nents and a part of AC components obtained as a result 
of the analysis at said frequency analyzing step as a still 

30 picture feature data and a picture feature data of a pic- 
ture within a picture data base where a still picture fea- 
ture data having been generated at said still picture 
feature extraction step on the basis of a still picture data 
is stored together with said still picture data; a similar 

35 still picture retrieving step for retrieving the picture within 
: said picture data base having a high correlation value 
with respect to the picture feature data generated on the 
basis of the inputted picture data as a candidate of sim- 
ilar still picture; a second picture feature data correlation 

40 measuring step for calculating a correlation value 
between a moving picture feature data of a moving pic- 
ture in inputted moving picture data generated by a 
moving picture feature extraction step including a fea- 
ture data extracting frame selecting step for taking out 

45 some data corresponding to the entire frames or a por- 
tion of the frames from the picture data identified as 
moving picture data, selecting the data subjected to 
processing, a resized picture generating step for gener- 
ating a resized picture from data corresponding to each 

so frame having been taken out, a frequency analyzing 
step for executing a frequency analysis on said resized 
picture having been generated by said resized picture 
generating step, a DC/partial AC component extracting 
step for taking out DC components and a part of AC 

55 components obtained as a result of the analysis at said 
frequency analyzing step as a frame feature data, and a 
moving picture feature data producing step for generat- 
ing a moving picture feature data by collecting said 
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frame feature data outputted at said DC/partial AC com- 
ponent extracting step, and a picture feature data of a 
picture within a picture data base where a moving pic- 
ture feature data having been generated at said moving 
picture feature extraction step on the basis of a moving 5 
picture data is stored together with said moving picture 
data; and a similar moving picture retrieving step for 
retrieving a moving picture segment within said picture 
data base having a high correlation value with respect 
to the moving picture feature data generated on the 10 
basis of the inputted moving picture data as a candidate 
of similar moving picture. 

[0035] In accordance with a twenty-ninth aspect of 
the present invention, there is provided a picture retriev- 
ing method wherein said second picture feature data is 
correlation measuring step has a step of dividing the 
moving picture feature data having been generated by 
said moving picture feature extraction step on the basis 
of said inputted moving picture data, and a step of cal- 
culating in turn a partial correlation value between a 20 
part of the moving picture feature data of the moving 
picture included in said moving picture data base and a 
moving picture feature data of each division of the 
divided moving picture feature data, and said similar 
picture retrieving step eliminates the moving picture 25 
segment included in said moving picture data base from 
the candidates of similar moving pictures when it has a 
feature data with a partial correlation value lower than a 
predetermined threshold value. 

[0036] In accordance with a thirtieth aspect of the 30 
present invention, there is provided a picture retrieving 
method wherein said second picture feature data corre- 
lation measuring step has a step of dividing the moving 
picture feature data having been generated by said 
moving picture feature extraction step on the basis of 35 
said inputted moving picture data, and a step of calcu- 
lating in turn a partial correlation value between a part 
of the moving picture feature data of the moving picture 
included in said moving picture data base and a partial 
moving picture feature data of each division of the 40 
divided moving picture feature data, and said similar 
picture retrieving step eliminates the moving picture 
segment included in said moving picture data base from 
the candidates of similar moving pictures at the time 
when it is acknowledged that a number of partial moving 45 
picture feature data having a feature data with a partial 
correlation value lower than a predetermined threshold 
value surpasses a predetermined threshold value. 
[0037] In accordance with a thirty-first aspect of the 
present invention, there is provided a picture retrieving so 
method wherein said resized picture generating step 
divides a picture into blocks, and takes out from each 
block color information representative of each block. 
[0038] In accordance with a thirty-second aspect of 
the present invention, there is provided a picture retriev- 55 
ing method wherein said resized picture generating step 
takes out DC components from said each block when 
said picture data is compressed by blocks. 



BRIEF DESCRIPTION OF THE DRAWINGS 

[0039] The above and further objects and the novel 
feature of the invention will more fully appear from the 
following detailed description when the same is read in 
connection with the accompanying drawings, in which: 

Fig. 1 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture feature extraction device and a 
method thereof are used; 

Fig. 2 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture feature extraction device and a 
method thereof are used; 

Fig. 3 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture feature extraction device and a 
method thereof are used; 

Fig. 4 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture feature extraction device and a 
method thereof are used; 

Fig. 5 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture feature extraction device and a 
method thereof are used; 

Fig. 6 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture retrieving device and a method 
thereof are used; 

Fig. 7 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture retrieving device and a method 
thereof are used; 

Fig. 8 is a flow chart showing an example of 
processing in the picture retrieving method of the 
present invention; 

Fig. 9 is a block diagram showing an example of 
structure of an embodiment of the present invention 
where a picture retrieving device and a method 
thereof are used; and 

Fig. 10 is a table showing an example of retrieval 
result of the present invention and grid layout based 
method. 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

[0040] Referring now to the drawings, a detail 
description of preferred embodiments of the present 
invention will be described in detail. 
[0041] In Fig 1 to Fig. 10, the embodiments of the 
picture feature extraction device, picture retrieving 
device, and methods of generating picture feature data 
and retrieving pictures, respectively, are illustrated. 
[0042] A first embodiment of the picture feature 
extraction device and method thereof in accordance 
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with the present invention will be described in detail with 
reference to Fig. 1. 

[0043] The picture feature extraction device in the 
first embodiment as shown in Fig. 1 comprises a still 
picture feature extraction section 22 for inputting still s 
picture data and generating the feature data of that still 
picture data having been inputted. The still picture fea- 
ture extraction section 22 has a resized picture generat- 
ing means 221, a frequency analyzing means 222, and 
a DC/partial AC component extracting means 223. w 
[0044] The resized picture generating means 221 
generates a resized picture 12 on the basis of an input 
of still picture data 11. The resized picture at this time 
can take any size. As to an example of a method of pro- 
ducing a resized picture, there is a method in which a 75 
picture is divided into blocks, and only color data repre- 
sentative of each block is taken out. In this case, as to 
the representative color data for each block, an average 
color, the most frequently used color, etc., can be used. 
Furthermore, in case when the picture data is com- 20 
pressed by blocks, it is possible to produce a resized 
picture by taking out the DC components from each 
block, and it is possible to produce a resized picture 
from said resized picture produced by taking out the DC 
components from each block. Any resolutions of the 25 
input picture can be allowed by resizing the input picture 
to a fixed size. 

[0045] The frequency analyzing means 222 con- 
ducts a frequency analysis on the resized picture 12 so 
as to calculate frequency components 13. It is also pos- 30 
sible to divide the resized picture 12 into a plurality of 
areas before executing the frequency analysis. As to a 
method of frequency analysis of the resized picture, 
DCT transformation, Wavelet transformation, Fourier 
transformation, Hadamard transformation, and so forth 35 
can be used. 

[0046] The DC/partial AC component extracting 
means 223 extracts DC/partial AC components 14 from 
the frequency components 13 having been calculated 
by the frequency analyzing means 222 to generate a 40 
feature data. As to a specific example of the application 
of the frequency analyzing means 222 and the DC/par- 
tial AC component extracting means 223, the frequency 
analysis can be executed by a two-dimensional DCT 
with respect to the whole size of the resized picture, so 45 
as to extract the DC components as well as n AC com- 
ponents from the calculated frequency components. 
Furthermore, it is also possible to extract only the lumi- 
nance coefficients, or extract only the chrominance 
coefficients, or part of both kinds of coefficients. so 
[0047] In accordance with the above-described pic- 
ture feature extraction device and method thereof, the 
frequency analysis is executed on the resized picture, 
and only a part of the acquired coefficients are used, 
whereby picture feature data requiring less memory 55 
capacity can be made available. 
[0048] A second embodiment of the picture feature 
extraction device and method thereof in accordance 



with the present invention will be described in detail with 
reference to Fig. 2. 

[0049] The picture feature extraction device in the 
second embodiment as shown in Fig. 2 comprises a 
moving picture feature extraction section 2 for inputting 
moving picture data and generating the feature data of 
that moving picture data having been inputted. The 
moving picture feature extraction section 2 has a feature 
data extracting frame selecting means 21, a still picture 
feature extraction section 22, and a feature data gener- 
ating means 23. As already mentioned, the still picture 
feature extraction section 22 has a resized picture gen- 
erating means 221 , a frequency analyzing means 222, 
and a DC/partial AC component extracting means 223. 
Moving picture by definition is made of a plurality of still 
pictures, and therefore, even when the subject here in 
the second embodiment is moving pictures, it is possi- 
ble to use the still picture feature extraction section 22 in 
this embodiment as in the first embodiment. 
[0050] The feature data extracting frame selecting 
means 21 as being a constituent of the moving picture 
feature extraction section 2 selects frame data 1 1 sub- 
jected to extraction of feature data on the basis of input- 
ted moving picture data 1 . 

[0051] The feature data extracting frame selecting 
means 21 can take out the entire frames or a portion of 
the frames in the moving picture data. The feature data 
extracting frame selecting means can also take out only 
the compressed data corresponding to an interframe 
encoded frame (l-picture) of which compression proc- 
ess is completed within a single frame in the moving pic- 
ture data. It is also possible to select the all l-Pictures or 
only a part of the l-Pictures. Furthermore, as to forward 
prediction encoded frames and bi-directional prediction 
encoded frames of which compression process is not 
completed within a frame, it is possible to produce the I- 
Picture in a pseudo-manner, using the method as intro- 
duced in "On the extraction of DC sequence from 
MPEG Compressed Video" by Boon-Lock Yeo and 
Bede Liu (IEEE International Conference on Image 
Processing, 1995). In this way, it is possible to produce 
the l-Picture in a pseudo- manner, and select them. 
[0052] The resized picture generating means 221 
constituting the still picture feature extraction section 22 
generates a resized picture on the basis of the still pic- 
ture data 11 selected by the feature data extracting 
frame selecting means 21. The frequency analyzing 
means 222 decomposes the resized picture 12 into fre- 
quency components 13. The DC/partial AC component 
extracting means 223 extracts the DC/jpartial AC com- 
ponents 14 from the frequency components 13 having 
been given by the frequency analyzing means 222. 
[0053] The feature data generating means 23 gen- 
erates feature data 15 by extracting the DC/partial AC 
components 14 given by the still picture feature extrac- 
tion section 22 with respect to the whole selected 
frames. 

[0054] In accordance with this embodiment also, in 
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the picture feature extraction device and method 
thereof, the frequency analysis is executed on the 
resized picture, and only a part of the acquired coeffi- 
cients are used, whereby picture feature data requiring 
less memory capacity can be made available. 5 
[0055] A third embodiment of the picture feature 
extraction device and method thereof in accordance 
with the present invention will be described in detail with 
reference to Fig. 3. 

[0056] The picture feature extraction device in the 10 
third embodiment as shown in Fig. 3 comprises a still 
picture feature extraction section 22 for inputting still 
picture data and generating the feature data of that still 
picture data having been inputted, and a feature data 
added still picture data base storing section 6. The fea- 75 
ture data added still picture data base storing section 6 
has a still picture data storing section 61 for storing the 
inputted still picture data, and a still picture feature data 
storing section 62 for storing the still picture feature data 
having been generated at the still picture feature extrac- 20 
tion section 22. The still picture feature extraction sec- 
tion 22 has a resized picture generating means 221, a 
frequency analyzing means 222, and a DC/partial AC 
component extracting means 223. 

[0057] In the still picture feature extraction section 25 
22, the resized picture generating means 221 generates 
a resized picture on the basis of the inputted still picture 
data, the frequency analyzing means 222 conducts a 
frequency analysis of the resized picture having been 
generated by the resized picture generating means 221 , 30 
and the DC/partial AC component extracting means 223 
takes out the DC components and a part of AC compo- 
nents obtained as a result of frequency analysis. Thus 
the still picture feature extraction section 22 generates 
the still picture feature data. 35 
[0058] The still picture feature data having been 
generated is to be stored in the still picture feature data 
storing section 62, while the still picture data used in 
generating the still picture feature data is to be stored in 
the still picture data storing section 61 . 40 
[0059] In accordance with this embodiment also, in 
the picture feature extraction device and method 
thereof, the frequency analysis is executed on the 
resized picture, and only a part of the acquired coeffi- 
cients are used, whereby picture feature data requiring 45 
less memory capacity can be made available. 
[0060] As shown in Fig. 4, the picture feature 
extraction device in a fourth embodiment of the picture 
feature extraction device and method thereof, has a 
moving picture feature extraction section 2, and a fea- so 
ture data added moving picture data base storing sec- 
tion 3. As to the moving picture feature extraction 
section 2, the description thereof will be omitted. 
[0061 ] In this embodiment, feature data 1 5 is gener- 
ated from moving picture data 1, and both the feature 55 
data 15 and the moving picture data 1 are stored in the 
feature data added moving picture data base storing 
section 3. 



[0062] The feature data added moving picture data 
base storing section 3 is composed of a moving picture 
data storing section 31, and a moving picture feature 
data storing section 32. With respect to various moving 
picture data 1, the feature data added moving picture 
data base storing section 3 stores both the moving pic- 
ture data 1 and the feature data 15 having been gener- 
ated by the feature extraction section 2 on the basis of 
the moving picture data 1 . 

[0063] The moving picture data storing section 31 
which is one of the two constituents of the feature data 
added moving picture data base storing section 3 stores 
the moving picture data 1. The moving picture feature 
data storing section 32 stores the feature data 15 hav- 
ing been generated by the feature extraction section 2 
on the basis of the moving picture data 1 . 
[0064] In accordance with the picture feature 
extraction device comprising the above mentioned com- 
ponents, data corresponding to the entire frames or a 
portion of the frames is taken out from the moving pic- 
ture data by the feature data extracting frame selecting 
means 21. Then the resized picture generating means 
221 generates a resized picture from the data corre- 
sponding to each frame being taken out. Then the fre- 
quency analyzing means 222 executes the frequency 
analysis of the resized picture having been produced by 
the resized picture generating means 221. Then the 
DC/partial AC component extracting means 223 takes 
out the DC components and a part of AC components 
obtained as a result of the frequency analysis, and the 
feature data generating means 23 is to execute the 
process with respect to the whole selected frames in 
order to generate the feature data 15. 
[0065] In the picture feature extraction device of this 
embodiment, the frequency analysis is executed on the 
resized picture of part of the frames, and only a part of 
the acquired coefficients are used, whereby picture fea- 
ture data requiring less memory capacity can be made 
available. 

[0066] A fifth embodiment of the picture feature 
extraction device and method thereof in accordance 
with the present invention will be described in detail with 
reference to Fig. 5. 

[0067] As illustrated in Fig. 5, the picture feature 
extraction device ol the fifth embodiment comprises a 
picture feature extraction section 10 and a feature data 
added picture data base storing section 5. The picture 
feature extraction section 10 comprises a picture data 
identifying section 100, a still picture feature extraction 
section 22, and a moving picture feature extraction sec- 
tion 2. The picture data in this embodiment is defined as 
including the moving picture data and the still picture 
data, and the moving picture data would be composed 
of a plurality of still pictures. 

[0068] In this embodiment, as the picture data iden- 
tifying section 100 receives the picture data 7, it deter- 
mines whether the inputted picture data is of still 
pictures or moving pictures. In case when the picture 
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data is of still pictures, the picture data is inputted to the 
still picture feature extraction section 22, while in case 
when the picture data is of moving pictures, the picture 
data is inputted to the moving picture feature extraction 
section 2. 5 
[0069] In the still picture feature extraction section 
22, the resized picture generating means 221 generates 
a resized picture on the basis of the inputted still picture 
data, the frequency analyzing means 222 conducts a 
frequency analysis on the resized picture having been io 
generated by the resized picture generating means 221 , 
and the DC/partial AC component extracting means 223 
takes out the DC components and a part of AC compo- 
nents obtained as a result of the frequency analysis, 
thus generating the feature data. 15 
[0070] In the moving picture feature extraction sec- 
tion 2, data corresponding to the entire frames or a por- 
tion of the frames is taken out from the moving picture 
data by the feature data extracting frame selecting 
means 21. Then the resized picture generating means 20 
221 generates a resized picture from the data corre- 
sponding to each frame being taken out. Then the fre- 
quency analyzing means 222 executes the frequency 
analysis on the resized picture having been produced 
by the resized picture generating means 221 . Then the 25 
DC/partial AC component extracting means 223 takes 
out the DC components and a part of AC components 
obtained as a result of the frequency analysis, and the 
feature data generating means 23 is to execute the 
process with respect to the entire frames being selected 30 
in order to generated the feature data. 
[0071] In accordance with the above-mentioned 
process, the feature data from the still picture feature 
extraction section and the feature data 15 from the mov- 
ing picture feature extraction section 2 are each stored 35 
in a picture feature data storing section 52 of the feature 
data added picture data base storing section 5. Further- 
more, the picture data composed of moving and still pic- . 
tures, respectively are stored in a picture data storing 
section 51 . 40 
[0072] Thus, when the still picture is inputted, the 
frequency analysis is executed on the resized picture, 
and only a part of the acquired coefficients are used in 
generating the feature data, while when the moving pic- 
ture is inputted, the frequency analysis is executed on 45 
the resized picture of a part of the frames, and only a 
part of the acquired coefficients are used in generating 
the feature data. Therefore, the still picture feature data 
as well as moving picture feature data requiring less 
memory capacity can be made available. so 
[0073] Referring now to Fig. 6, a first embodiment 
of the picture retrieving device and method thereof will 
be described. 

[0074] As shown in Fig. 6. in this embodiment, the 
picture retrieving device comprises a still picture feature 55 
extraction section 22 for inputting still picture data and 
generating the feature data of that still picture data hav- 
ing been inputted, a feature data added still picture data 



base storing section 6 including a still picture data stor- 
ing section 61 for storing the inputted still picture data 
and a still picture feature data storing section 62 for stor- 
ing the still picture feature data having been generated 
from the still picture feature extraction section 22, and a 
still picture correlation measuring section 41 for calcu- 
lating a correlation value between the still picture fea- 
ture data having been generated from the inputted still 
picture data by the still picture feature extraction section 
22 and the still picture feature data stored in the feature 
data added still picture data base storing section 6. 
[0075] In the still picture feature extraction section 
22, the resized picture generating means 221 generates 
a resized picture on the basis of the inputted still picture 
data, the frequency analyzing means 222 conducts a 
frequency analysis on the resized picture having been 
generated by the resized picture generating means 221 , 
and the DC/partial AC component extracting means 223 
takes out the DC components and a part of AC compo- 
nents obtained as a result of the frequency analysis. 
Thus the still picture feature extraction section 22 gener- 
ates the still picture feature data with respect to the 
inputted still picture data 1 1 . 

[0076] In the feature data added still picture data 
base storing section 6, through the same process as 
described in the above third embodiment of the picture 
feature extraction device, the still picture has been 
stored in the still picture data storing section 61 , and the 
still picture feature data calculated by the still picture 
feature extraction section 22 using the still picture 
stored in the still picture data storing section 61 has 
been stored in the still picture feature data storing sec- 
tion 62. 

[0077] The still picture correlation measuring sec- 
tion 41 is to calculate the correlation value between the 
feature data having been generated from the inputted 
still picture data by the still picture feature extraction 
section 22 and the feature data having been generated 
from the picture included in the picture data base. Then 
the still picture correlation measuring section 41 is to 
retrieve the still picture in the data base having a high 
correlation value as a candidate of a similar still picture. 
[0078] Referring now to Fig. 7 and Fig. 8, a second 
embodiment of the picture retrieving device and method 
thereof will be described. Fig. 7 is a block diagram 
showing the structure of the picture retrieving device in 
the second embodiment of the picture retrieving device 
and method thereof. Fig. 8 is a flow chart showing a 
process of correlation measurement in the picture 
retrieving method. 

[0079] As shown in Fig. 7, the picture retrieving 
device in the second embodiment comprises a feature 
extraction section 2, a feature data added moving pic- 
ture data base storing section 3, and a correlation 
measuring section 4 for calculating a correlation value 
between feature data 1 7 in the moving picture data base 
and feature data 15 having been generated by the fea- 
ture extraction section 2 on the basis of the inputted 
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moving picture data 1 . 

[0080] First, the moving picture data to be used for 
a retrieval is inputted to the feature extraction section 2, 
from which the feature data 15 is generated. In execut- 
ing the retrieval, feature data is generated form the ref- 
erence moving picture, so as to be inputted to the 
correlation measuring section 4. 
[0081] The correlation measuring section 4 calcu- 
lates a correlation value between the feature data 17 in 
the moving picture data base and the feature data 15 
having been generated by the feature extraction section 
2 on the basis of the inputted moving picture data 1. 
Then storage location data 18 of a partial moving pic- 
ture being retrieved is outputted as a retrieving result. 
[0082] The operation of the moving picture correla- 
tion measuring section 4 will be described in detail with 
reference to Fig. 8. 

[0083] In accordance with the picture retrieving 
method of the present invention, the processing proc- 
ess for obtaining the correlation value between the fea- 
ture data having been generated from the inputted 
moving picture data and a part of the feature data hav- 
ing been generated from the moving picture included in 
the moving picture data base consists of three steps in 
which the feature data having been generated from the 
inputted moving picture is divided in a first step, partial 
correlation is calculated in sequence for each of the 
divided parts of the feature data in a second step, and 
the moving picture in the data base having the feature 
data corresponding to the partial correlation lower than 
the predetermined threshold value is eliminated from 
the candidate pictures in a third step. By adapting such 
method, the candidate pictures as a result of a retrieval 
can be narrowed down in number through a less 
number of matching process, as compared to the case 
in which the matching process is executed on the entire 
designated moving picture segments in a round robin 
manner. This contributes to shortening the processing 
time required in retrieving the moving pictures. The 
details of this processing process will be described in 
the following. 

[0084] First, in the initial setting, identification (ID) 
numbers for bringing out the moving picture feature data 
subjected to retrieval is initialized (step S1). In this case, 
the ID number is a unique number given individually to 
each moving picture data in generating the moving pic- 
ture data base. The feature data 15 should also be pre- 
viously acquired. 

[0085] Next, the moving picture feature data sub- 
jected to retrieval, which corresponds to the ID is 
acquired, and a pointer indicating the acquisition loca- 
tion of the partial feature data subjected to retrieval (to 
be referred to as target partial feature data) is arranged 
in the front (step S2). 

[0086] The feature data of a predetermined time 
length is acquired from the reference partial feature 
data in the front. This data is to be referred to as refer- 
ence partial feature data. Furthermore, the target partial 



feature data is acquired from the point indicated by the 
pointer showing the acquisition location (step S3). 
[0087] Then the correlation between the reference 
partial feature data and the target partial feature data 

s acquired in step S3 will be determined (step S4). 

[0088] When the acquired correlation value is lower 
than the predetermined threshold value, (the condition 
will be referred to as a mismatch judgement), the pres- 
ence of the target partial feature data to be acquired 

w next will be checked in step S5. When the presence of 
the next target feature data is acknowledged, the acqui- 
sition location pointer of the target partial feature data is 
shifted by a predetermined value (step S6), before 
returning to step S3. On the other hand, when the next 

is target partial feature data is not there, another ID is 
added, and the same process is conducted on the fol- 
lowing moving picture feature data (step S13). However, 
in case when the matching process is conducted on the 
entire moving picture feature data within the data base, 

20 the process is to be terminated (step S12). 

[0089] In case when the correlation value is deter- 
mined as larger than the predetermined threshold 
value, the next target partial feature data and the refer- 
ence partial feature data following the acquisition loca- 

25 tion followed by a certain time length is acquired (step 
S7). and the correlation will be determined (step S8). As 
to the subsequent events, the matching process is 
repeated in the same manner until encountering a mis- 
match, and when the entire reference partial feature 

30 data match (step S9), the result will be outputted as a 
retrieving result (step S10). 

[0090] In this occasion, it is possible to output the 
moving picture data ID number including the retrieving 
result, the start location, the end location, etc. in the 

35 moving picture data, correlation value, etc. 

[0091] In case when the mismatch judgement is 
given, it is to be determined whether a predetermined 
number of mismatch judgements or a predetermined 
period of time having been separately set has sur- 

40 passed (step S11). If not surpassed, the process 
returns to step S7. If surpassed, the process returns to 
step S5. In step S1 1 , temporal mismatches caused by a 
difference in a temporal position of a frame, a difference 
in camera work, a change in the feature data due to 

45 screen noises, etc. can be resolved by appropriately 
setting the number of times and a period of time with 
respect to the mismatch judgements. Furthermore, set- 
ting the number of times and a period of time to "0" is 
the same as eliminating step S11, and thus it is also 

so possible to simplify the circuit design. 

[0092] On the other hand, it is also possible to con- 
sider the time length in acquiring the reference partial 
feature data as of the entire reference feature data, so 
as to compare the correlation with the retrieving partial 

55 feature data one at a time, thus executing the retrieval. 
[0093] A third embodiment of the picture retrieving 
device and method thereof in accordance with the 
present invention will be described in detail with refer- 
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ence to Fig. 9. 

[0094] As shown in Fig. 9, the picture retrieving 
device of this embodiment comprises a picture feature 
extraction section 10, a feature data added picture data 
base storing section 5 including a picture data storing s 
section 51 for storing still pictures and moving pictures 
and a picture feature data storing section 52 storing the 
still picture feature data and the moving picture feature 
data having been previously generated by the picture 
feature extraction section 10 on the basis of the picture 10 
data stored in the picture data storing section, a moving 
picture correlation measuring section 4 for calculating a 
correlation value between feature data 1 7 in the moving 
picture data base and feature data 15 having been gen- 
erated by the feature extraction section 2 with respect to is 
the inputted moving picture data 1, and a still picture 
correlation measuring section 41 for calculating the cor- 
relation value between the feature data 1 7 in the picture 
data base and the feature data having been generated 
by the picture feature extraction section 1 0 on the basis 20 
of the inputted still picture data 1 1 . 
[0095] First, picture data 1 used in a retrieval is 
inputted to generate the feature data 15 by the feature 
extraction section 2. In case when the picture data for 
reference is of moving pictures, feature data is gener- 25 
ated from the moving picture by the feature extraction 
section 10 to be inputted to the moving picture feature 
data correlation measuring section 4. In case when the 
picture data for reference is of still pictures, feature data 
is generated from the still picture by the feature extrac- 30 
tion section 10 to be inputted to the still picture feature 
data correlation measuring section 41. 
[0096] The moving picture correlation measuring 
section 4 is to calculate the correlation value between 
the feature data 17 within the moving picture data base 35 
and the feature data 1 5 generated by the feature extrac- 
tion section 2 on the basis of the inputted moving picture 
data 1 . Then it outputs storage location data 18 on the 
partial moving picture having been retrieved as a 
retrieving result. The explanation on the processing 40 
operation in this embodiment will be omitted for the 
explanation has already been given in the description of 
the second embodiment of the moving picture feature 
extraction device. 

[0097] The still picture correlation measuring sec- 45 
tion 41 is to calculate the correlation value between the 
feature data having been generated from the inputted 
still picture data by the feature extraction section 1 0 and 
the feature data generated by the picture included in the 
picture data base, so as to retrieve a still picture within so 
the data base having a high correlation value as a can- 
didate for a similar still picture. 

[0098] In the following, an example of an embodi- 
ment of the present invention will be described. 
[0099] An experiment is conducted using 50 que- 55 
ries among 5466 still pictures, where the correct results 
for those 50 queries are pre-defined by human judge- 
ment based on similarity. 



[01 00] First, all still pictures are divided into 64 (8 x 
8) blocks and average color of each block is taken out. 
The derived average colors are transformed into a 
series of coefficients by 8 x 8 DCT Three DC coeffi- 
cients, five luminance AC coefficients and two compo- 
nents per each chrominance AC coefficients normalized 
within the range of [0, 255] for each component are 
used as feature data. 

[0101] Figure 10 shows a comparison between 
using DCT coefficients on a picture resized to 8 x 8 pix- 
els and 2x2, 3x3, 4x4 grid layout and average color 
within the range of [0, 255] for each color component as 
feature data. Correct retrieval rate means how many of 
the correct pictures are detected within top K ranks 
when all the images are sorted according to a weighted 
Euclidean distance between feature data of a query pic- 
ture and stored pictures. Here K denotes a number of 
correct results multiplied by four. Figure 10 indicates 
that DCT coefficients perform better than any grid 
based feature data with smaller size of feature data. 
[0102] In the following, another example of an 
embodiment of the present invention will be described. 
[0103] An experiment is conducted using 20 hours 
worth of video data being encoded by MPEG-1 , MPEG- 
2 encodings, where a certain video segment corre- 
sponding to a particular article, TV commercial, and an 
opening of a particular program is designated in retriev- 
ing a video segment seemingly identical to the desig- 
nated video segment. 

[0104] First, a time code and DC components are 
extracted from each compressed data corresponding to 
l-Picture within the moving picture as a whole subjected 
to retrieval. In this occasion, the resized picture having 
been generated by extracting the DC components is fur- 
ther resized to a fixed size of 8 pixel x 8 pixel and 
exposed to DCT to obtain three DC luminance and 

chrominance components (to be referred to as DC Y, 

DC Cb, and DC Cr), as well as 5 luminance AC 

components, 2 components per each chrominance AC 

component (to be referred to as AC Y AC Cb, and 

AC Cr), which are normalized within the range of [0, 

255] for each DC and AC component and stored 
together with the time codes, respectively. For each I- 
Picture, only a total of 1 6 bytes including 12 bytes of fea- 
ture dada and 4 bytes of time code are required. There- 
fore, when the l-Pictures exist in the intervals of 0.5 
seconds, the size of data of the feature data corre- 
sponding to the amount of video worthy of a portion for 
a single day can be managed within about 2.8 mega 
bytes. 

[01 05] The feature data of the head frame of the ref- 
erence moving picture goes through a matching proc- 
ess with the feature data of the retrieving target moving 
picture in a unit of l-Picture. A candidate for the aiming 
moving picture is defined by determining the ones of 

which difference value of DC Y, DC_Cb, and DC Cr 

(to be referred to as DC difference value), and the differ- 
ence value of AC Y f AC Cb and AC Cr meet a cer- 
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tain specific condition (threshold value). 
[0106] When a certain frame is determined as a 
candidate for the aiming moving picture, the frames 
ahead are to be inspected in time series. In case when 
the frame is not determined as a candidate for the aim- 5 
ing moving picture, the same process as described is to 
be repeated on the next frame. After that, the ones of 
which DC difference values meet the specified condition 
are to be regarded as the matched ones, and the 
inspection is to be conducted in due order. For the pur- 10 
pose of reducing the number of times conducting the 

matching process, AC Y, AC Cb, and AC_Cr can be 

disregarded. 

[0107] Considering the errors due to sampling 
points, when the mismatch is determined three frames 15 
in a row, the frames are eliminated from the candidates. 
When the difference of feature data with respect to a 
length corresponding to the designated moving picture 
segment meets the specified condition (threshold 
value), it is to be determined as the aiming moving pic- 20 
ture segment, in response to which the time codes for 
the start point and the end point of that certain length 
are outputted as a retrieving result. Furthermore, in 
case when the aiming moving picture segment is 
retrieved, the start point for the next searching can be 25 
set as the frame next to the end point of the detected 
section. 

[0108] According to such process, compared to the 
case in which the matching process is conducted in a 
round robin manner over the entire designated moving 30 
picture segment, it is possible to reduce the amount of 
processing by a considerable extent. 
[0109] As an experiment was conducted on 11 
pairs of moving picture segments which seem identical, 
it was proved with the whole 1 1 pairs that when one of 35 
the two moving picture segments in a single pair is des- 
ignated, it was possible to obtain the other one in the 
pair. Furthermore, even when the length of the aiming 
moving picture segment was set as over 3 seconds, no 
excessive retrieving was noted. 40 
[0110] The above described embodiments are 
merely an example of the preferred embodiment of the 
present invention. The present invention, therefore, is 
not limited to such illustrative description but changes 
and variations may be made without departing from the 45 
subject matter of the invention. In any of the above 
described embodiments, the application of the present 
invention can be extended to a retrieving of similar mov- 
ing picture segments by defining the distance between 
the feature data instead of excluding the ones not meet- so 
ing the condition. 

[0111] As it has been made clear in the above 
description, in accordance with the picture feature 
extraction device and method thereof of the present 
invention, the feature data is generated through gener- 55 
ating a resized picture from the picture data, executing a 
frequency analysis on the generated resized picture, 
and taking out DC components and a part of AC compo- 



nents obtained as a result of the analysis. 
[01 12] Furthermore, in accordance with the picture 
feature extraction device and method thereof, data cor- 
responding to the entire frames or part of the frames in 
the picture data is taken out, and the resized picture is 
generated from the data corresponding to each frame 
being taken out. The frequency analysis is conducted 
on the generated resized picture, the DC components 
and a part of AC components obtained as a result of the 
frequency analysis are taken out, and this process is 
implemented on the whole frames being selected. 
[0113] In accordance with the picture retrieving 
device and method thereof of the present invention, it is 
possible to eliminate the pictures from the candidates 
when it becomes obvious that the picture would have no 
chance of being identified as a similar picture in the 
process of calculating the correlation. According to such 
process, compared to the case in which the matching 
process is conducted in a round robin manner over the 
entire designated moving picture segment, it is possible 
to reduce the number of matching process for narrowing 
down the retrieving candidates to be regarded as the 
retrieving result, thus reducing the amount of process- 
ing by a considerable extent. 

[01 14] Furthermore, in accordance with the present 
invention, the frequency analysis is executed on the 
resized picture, and only a part of the acquired coeffi- 
cients are used, whereby picture feature data requiring 
less memory capacity can be made available. 
[01 15] The present invention can widely apply to a 
variety of purposes such as an investigation on a broad- 
casting frequency, obtaining a statistics on some broad- 
casting information, a tendency analysis, etc., for a 
particular video image such as TV commercial pictures, 
news articles, opening of particular programs, etc. 
[0116] Furthermore, the present invention can 
apply to a retrieving of similar picture etc., for retrieving 
a similar color layout picture for reference within the 
stored pictures. 

[0117] While the preferred embodiments of the 
invention have been described using specific terms, 
such description is for illustrative purposes only, and it is 
to be understood that changes and variations may be 
made without departing from the spirit or the scope of 
the following claims. 

Claims 

1 . A picture feature extraction device comprising: 

a resized picture generating means generating 
a resized picture from still picture data; 
a frequency analyzing means executing a fre- 
quency analysis on said resized picture having 
been generated by said resized picture gener- 
ating means; and 

a DC/partial AC component extracting means 
taking out DC components and a part of AC 
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components obtained as a result of the analy- 
sis by said frequency analyzing means as a 
picture feature data. 

2. A picture feature extraction device as claimed in 5 
claim 1, comprising: 

a still picture data base including a still picture 
data storing means storing said still picture 
data, and a still picture feature data storing 10 
means storing said still picture feature data 
generated for each still picture data stored in 
said still picture data storing means. 

3. A picture feature extraction device comprising: 15 

a feature data extracting frame selecting 
means taking out some data corresponding to 
the entire frames or a portion of the frames 
from the moving picture data, selecting the 20 
data subjected to processing; 
a resized picture generating means generating 
a resized picture from data corresponding to 
each frame having been taken out; 
a frequency analyzing means executing a fre- 25 
quency analysis on said resized picture having 
been generated by said resized picture gener- 
ating means; 

a DC/partial AC component extracting means 
taking out DC components and a part of AC 30 
components obtained as a result of the analy- 
sis by said frequency analyzing means as a 
frame feature data; and 
a moving picture feature data producing means 
producing a moving picture feature data by col- 35 
lecting said frame feature data outputted from 
said DC/partial AC component extracting 
means. 

4. A picture feature extraction device as claimed in 40 
claim 3, comprising: 

a moving picture data base including a moving 
picture data storing means storing said moving 
picture data, and a moving picture feature data 45 
storing means storing said moving picture fea- 
ture data generated for each moving picture 
data stored in said moving picture data storing 
means. . . 

50 

5. A picture feature extraction device comprising: 

a picture data identifying means identifying 
whether the inputted picture data is of still pic- 
tures or of moving pictures; 55 
a still picture feature extraction means includ- 
ing a resized picture generating means gener- 
ating a resized picture from the picture data 



identified as still picture data by said picture 
data identifying means, a frequency analyzing 
means executing a frequency analysis on said 
resized picture having been generated by said 
resized picture generating means, and a 
DC/partial AC component extracting means 
taking out DC components and a part of AC 
components obtained as a result of the analy- 
sis by said frequency analyzing means as a still 
picture feature data; and 
a moving picture feature extraction means 
including a feature data extracting frame 
selecting means taking out some data corre- 
sponding to the entire frames or a portion of the 
frames from the picture data identified as mov- 
ing picture data by said picture data identifying 
means, selecting the data subjected to 
processing, a resized picture generating 
means generating a resized picture from data 
corresponding to each frame having been 
taken out, a frequency analyzing means exe- 
cuting a frequency analysis on said resized pic- 
ture having been generated, a DC/partial AC 
component extracting means taking out DC 
components and a part of AC components 
obtained as a result of the analysis by said fre- 
quency analyzing means as a frame feature 
data, and a moving picture feature data pro- 
ducing means generating a moving picture fea- 
ture data by collecting said frame feature data 
outputted from said DC/partial AC component 
extracting means. 

6. A picture feature extraction device as claimed in 
claim 5, comprising: 

a picture data base including a picture data 
storing means storing said picture data, and a 
picture feature data storing means using each 
picture data stored in said picture data storing 
means for storing the still picture feature data 
generated by said still picture feature extraction 
means and the moving picture feature data 
generated by said moving picture feature 
extraction means as a picture feature data. 

7. A picture feature extraction device as claimed in 
claim 2, wherein 

said resized picture generating means 
divides a still picture into blocks, and takes out color 
data representative of each block from each block. 

8. A picture feature extraction device as claimed in 
claim 4 or 6, wherein 

said resized picture generating means 
divides a still picture into blocks, and takes out color 
data representative of each block from each block 
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9. A picture feature extraction device as claimed in 
claim 2, 4, or 6 wherein 

in case when said picture data is com- 
pressed by blocks, said resized picture generating 
means takes out DC components from each block. 5 

10. A picture retrieving device comprising: 

a still picture generating means including a 
resized picture generating means generating a io 
resized picture from still picture data, a fre- 
quency analyzing means executing a fre- 
quency analysis on said resized picture having 
been generated by said resized picture gener- 
ating means, and a DC/partial AC component is 
extracting means taking out DC components 
and a part of AC components obtained as a 
result of the analysis by said frequency analyz- 
ing means as a picture feature data; 
a still picture data base including a still picture 20 
feature data storing means storing the still pic- 
ture feature data previously being generated by 
said still picture feature extraction means on 
the basis of the still picture data, and a still pic- 
ture data storing means storing said still picture 25 
data; and 

a still picture feature data correlation measur- 
ing means calculating a correlation value 
between the still picture feature data generated 
by said still picture feature extraction means on 30 
the basis of the inputted still picture data and 
the still picture feature data stored in said still 
picture data base, the still picture feature data 
correlation measuring means retrieving the still 
picture within the still picture data base having 35 
a high correlation value as a candidate of simi- 
lar still picture. 

1 1 . A picture retrieving device comprising: 

40 

a moving picture feature extraction means 
including a feature data extracting frame 
selecting means taking out some data corre- 
sponding to the entire frames or a portion of the 
frames from the moving picture data, selecting 45 
the data subjected to processing, a resized pic- 
ture generating means generating a resized 
picture from data corresponding to each frame 
having been taken out, a frequency analyzing 
means executing a frequency analysis on said so 
resized picture having been generated by said 
resized picture generating means, a DC/partial 
AC component extracting means taking out DC 
components and a part of AC components 
obtained as a result of the analysis by said fre- 55 
quency analyzing means as a frame feature 
data, and a moving picture feature data pro- 
ducing means producing a moving picture fea- 



ture data by collecting said frame feature data 
outputted from said DC/partial AC component 
extracting means; 

a moving picture data base including a moving 
picture feature data storing means storing the 
moving picture feature data previously being 
generated by said moving picture feature 
extraction means on the basis of the moving 
picture data, and a moving picture data storing 
means storing said moving picture data; and 
a moving picture feature data correlation meas- 
uring means calculating a correlation value 
between the moving picture feature data gener- 
ated by said moving picture feature extraction 
means on the basis of the inputted moving pic- 
ture data and the moving picture feature data 
stored in said moving picture data base, the 
moving picture feature data correlation meas- 
uring means retrieving the moving picture seg- 
ment within the moving picture data base 
having a high correlation value as a candidate 
of similar moving picture segment. 

12. A picture retrieving device comprising: 

a picture data identifying means identifying 
whether the inputted picture data is of still pic- 
tures or of moving pictures; 
a still picture feature extraction means includ- 
ing a resized picture generating means gener- 
ating a resized picture from the picture data 
identified as still picture data by said picture 
data identifying means, a frequency analyzing 
means executing a frequency analysis on said 
resized picture having been generated by said 
resized picture generating means, and a 
DC/partial AC component extracting means 
taking out DC components and a part of AC 
components obtained as a result of the analy- 
sis by said frequency analyzing means as a still 
picture feature data; 

a moving picture feature extraction means 
including a feature data extracting frame 
selecting means taking out some data corre- 
sponding to the entire frames or a portion of the 
frames from the picture data identified as mov- 
ing picture data by said picture data identifying 
means, selecting the data subjected to 
processing, a resized picture generating 
means generating a resized picture from data 
corresponding to each frame having been 
taken out, a frequency analyzing means exe- 
cuting a frequency analysis on said resized pic- 
ture having been generated by said resized 
picture generating means, a DC/partial AC 
component extracting means taking out DC 
components and a part of AC components 
obtained as a result of the analysis by said fre- 
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quency analyzing means as a frame feature 
data, and a moving picture feature data pro- 
ducing means generating a moving picture fea- 
ture data by collecting said frame feature data 
outputted from said DC/partial AC component 5 
extracting means; 

a picture data base including a picture feature 
data storing means storing the still picture fea- 
ture data previously being generated by said 
still picture feature extraction means on the 10 
basis of the still picture data and the moving 
picture feature data previously being generated 
by said moving picture feature extraction 
means on the basis of the moving picture data, 
and a picture data storing means storing said 75 
still picture data and said moving picture data; 
a still picture feature data correlation measur- 
ing means calculating a correlation value 
between the still picture feature data generated 
by said still picture feature extraction means on so 
the basis of the inputted still picture data and 
the picture feature data stored in said picture 
data base; and 

a moving picture feature data correlation meas- 
uring means calculating a correlation value 25 
between the moving picture feature data gener- 
ated by said moving picture feature extraction 
means on the basis of the inputted moving pic- 
ture data and the picture feature data stored in 
said picture data base, 30 
said still picture feature data correlation meas- 
uring means retrieving the still picture within 
the picture data base having a high correlation 
value as a candidate of similar still picture, 
said moving picture feature data correlation 35 
measuring means retrieving the moving picture 
segment within the picture data base having a 
high correlation value as a candidate of similar 
moving picture segment. 

40 

13. A picture retrieving device as claimed in claim 1 1 or 
12, wherein 

said moving picture feature data correlation 
measuring means divides the moving picture fea- 
ture data having been generated by said moving 45 
picture feature extraction means on the basis of 
said inputted moving picture data, and calculates in 
turn a partial correlation value between a part of the 
moving picture feature data of the moving picture 
included in said moving picture data base and a so 
moving picture feature data of each division of the 
divided moving picture feature data, upon which 
eliminating the moving picture segment included in 
said moving picture data base from the candidates 
of similar moving pictures when it has a feature data 55 
with a partial correlation value lower than a prede- 
termined threshold value. 



14. A picture retrieving device as claimed in claim 1 1 or 
12, wherein 

said moving picture feature data correlation 
measuring means divides the moving picture fea- 
ture data having been generated by said moving 
picture feature extraction means in respect with 
said inputted moving picture data, and calculates in 
turn a partial correlation value between a part of the 
moving picture feature data of the moving picture 
included in said moving picture data base and a 
partial moving picture feature data of each division 
of the divided moving picture feature data, upon 
which eliminating the moving picture segment 
included in said moving picture data base from the 
candidates of similar moving pictures at the time 
when it is acknowledged that a number of partial 
moving picture feature data having a feature data 
with a partial correlation value lower than a prede- 
termined threshold value surpasses a predeter- 
mined threshold value. 

15. A picture retrieving device as claimed in one of 
claims 10 to 14, wherein 

said resized picture generating means 
divides a picture into blocks, and takes out from 
each block color information representative of each 
block. 

16. A picture retrieving device as claimed in one of 
claims 10 to 14, wherein 

said resized picture generating means takes 
out DC components from said each block when 
said picture data is compressed by blocks. 

17. A picture feature extraction method comprising: 

a resized picture generating step for generating 
a resized picture from still picture data; 
a frequency analyzing step for executing a fre- 
quency analysis on said resized picture having 
been generated by said resized picture gener- 
ating step; and 

a DC/partial AC component extracting step for 
taking out DC components and a part of AC 
components obtained as a result of the analy- 
sis at said frequency analyzing step as a pic- 
ture feature data. 

18. A picture feature extraction method as claimed in 
claim 17, comprising: 

a still picture data storing step for storing said 
still picture data; and 

a still picture feature data storing step for stor- 
ing said still picture feature data generated for 
each still picture data stored at said still picture 
data storing step. 
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19. A picture feature extraction method comprising: 

a feature data extracting frame selecting step 
for taking out some data corresponding to the 
entire frames or a portion of the frames from 5 
the moving picture data, selecting the data sub- 
jected to processing; 

a resized picture generating step for generating 
a resized picture from data corresponding to 
each frame having been taken out; io 
a frequency analyzing step for executing a fre- 
quency analysis on said resized picture having 
been generated at said resized picture gener- 
ating step; 

a DC/partial AC component extracting step for 15 
taking out DC components and a part of AC 
components obtained as a result of the analy- 
sis at said frequency analyzing step as a frame 
feature data; and 

a moving picture feature data producing step 20 
for producing a moving picture feature data by 
collecting said frame feature data outputted 
from said DC/partial AC component extracting 
step. 



20. A picture feature extraction method as claimed in 
claim 19. comprising: 

a moving picture data storing step for storing 
said moving picture data; and 30 
a moving picture feature data storing step for 
storing said moving picture feature data gener- 
ated for each moving picture data stored at 
said moving picture data storing step. 

35 

21 . A picture feature extraction method comprising: 

a picture data identifying step for identifying 
whether the inputted picture data is of still pic- 
tures or of moving pictures; 40 
a still picture feature extraction step including a 
resized picture generating step for generating a 
resized picture from the picture data identified 
as still picture data at said picture data identify- 
ing step, a frequency analyzing step for execut- 45 
ing a frequency analysis on said resized picture 
having been generated at said resized picture 
generating step, and a DC/partial AC compo- 
nent extracting step for taking out DC compo- 
nents and a part of AC components obtained so 
as a result of the analysis at said frequency 
analyzing step as a still picture feature data; 
and 

a moving picture feature extraction step includ- 
ing a feature data extracting frame selecting 55 
step for taking out some data corresponding to 
the entire frames or a portion of the frames 
from the picture data identified as moving pic- 



ture data at said picture data identifying step, 
selecting the data subjected to processing, a 
resized picture generating step for generating a 
resized picture from data corresponding to 
each frame having been taken out, a frequency 
analyzing step for executing a frequency analy- 
sis on said resized picture having been gener- 
ated at said resized picture generating step, a 
DC/Partial AC component extracting step for 
taking out DC components and a part of AC 
components obtained as a result of the analy- 
sis at said frequency analyzing step as a frame 
feature data, and a moving picture feature data 
producing step for generating a moving picture 
feature data by collecting said frame feature 
data outputted at said DC/Partial AC compo- 
nent extracting step. 

22. A picture feature extraction method as claimed in 
claim 21 , comprising: 

a picture data storing step for storing said pic- 
ture data; and 

a picture feature data storing step using each 
picture data stored at said picture data storing 
step for storing the still picture feature data 
generated at said still picture feature extraction 
step and the moving picture feature data gen- 
erated at said moving picture feature extraction 
step as a picture feature data. 

23. A picture feature extraction method as claimed in 
claim 18, 20 or 21 , wherein 

said resized picture generating step divides 
a still picture into blocks, and takes out from each 
block color data representative of each block. 

24. A picture feature extraction method as claimed in 
claim 18, 20 or 21 , wherein 

in case when said picture data is com- 
pressed by blocks, said resized picture generating 
step takes out from each block DC components. 

25. A picture retrieving method comprising: 

a still picture feature extraction step including a 
resized picture generating step for generating a 
resized picture from still picture data, a fre- 
quency analyzing step for executing a fre- 
quency analysis on said resized picture having 
been generated at said resized picture gener- 
ating step, and a DC/partial AC component 
extracting step for taking out DC components 
and a part of AC components obtained as a 
result of the analysis at said frequency analyz- 
ing step as a picture feature data; 
a still picture feature data correlation measur- 
ing step for calculating a correlation value 
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between the still picture feature data of the still 
picture within a still picture data base where the 
still picture feature data of the still picture data 
having been previously generated at said still 
picture feature extraction step is stored 5 
together with the still picture data and the still 
picture feature data generated at said still pic- 
ture feature extraction step on the basis of the 
inputted still picture data; and 
a similar still picture retrieving step for retriev- 10 
ing the still picture within said still picture data 
base having a high correlation value with 
respect to the still picture feature data gener- 
ated on the basis of the inputted still picture 
data as a candidate of similar still picture. 15 

26. A picture retrieving method comprising: 

a moving picture feature extraction step includ- 
ing a feature data extracting frame selecting 20 
step for taking out some data corresponding to 
the entire frames or a portion of the frames 
from the moving picture data, selecting the 
data subjected to processing, a resized picture 
generating step for generating a resized picture 25 
from data corresponding to each frame having 
been taken out, a frequency analyzing step for 
executing a frequency analysis on said resized 
picture having been generated by said resized 
picture generating step, a DC/partial AC com- 30 
ponent extracting step for taking out DC com- 
ponents and a part of AC components obtained 
as a result of the analysis at said frequency 
analyzing step as a frame feature data, and a 
moving picture feature data producing step for 35 
producing a moving picture feature data by col- 
lecting said frame feature data outputted at 
said DC/partial AC component extracting step; 
a moving picture feature data correlation meas- 
uring step for calculating a correlation value 40 
between the moving picture feature data of the 
moving picture within a moving picture data 
base where the moving picture feature data of 
the moving picture data having been previously 
generated at said moving picture feature 45 
extraction step is stored together with the mov- 
ing picture data and the moving picture feature 
data generated at said moving picture feature 
extraction step on the basis of the inputted 
moving picture data; and so 
a similar moving picture retrieving step for 
retrieving the moving picture segment within 
said moving picture data base having a high 
correlation value with respect to the moving 
picture feature data generated on the basis of 55 
the inputted moving picture data as a candidate 
of similar moving picture. 



27. A picture retrieving method as claimed in claim 26, 
wherein 

said moving picture feature data correlation 
measuring step has a step of dividing the moving 
picture feature data having been generated by said 
moving picture feature extraction step on the basis 
of said inputted moving picture data, and a step of 
calculating in turn a partial correlation value 
between a part of the moving picture feature data of 
the moving picture included in said moving picture 
data base and a moving picture feature data of 
each division of the divided moving picture feature 
data, and 

said similar picture retrieving step eliminates 
the moving picture segment included in said mov- 
ing picture data base from the candidates of similar 
moving pictures when it has a feature data with a 
partial correlation value lower than a predetermined 
threshold value. 

28. A picture retrieving method as claimed in claim 26, 
wherein 

said moving picture feature data correlation 
measuring step has a step of dividing the moving 
picture feature data having been generated by said 
moving picture feature extraction step on the basis 
of said inputted moving picture data, and a step of 
calculating in turn a partial correlation value 
between a part of the moving picture feature data of 
the moving picture included in said moving picture 
data base and a partial moving picture feature data 
of each division of the divided moving picture fea- 
ture data, and 

said similar picture retrieving step eliminates 
the moving picture segment included in said mov- 
ing picture data base from the candidates of similar 
moving pictures at the time when it is acknowl- 
edged that a number of partial moving picture fea- 
ture data having a feature data with a partial 
correlation value lower than a predetermined 
threshold value surpasses a predetermined thresh- 
old value. 

29. A picture retrieving method comprising: 

a first picture feature data correlation measur- 
ing step for calculating a correlation value 
between a still picture feature data of a still pic- 
ture in inputted still picture data generated by a 
still picture feature extraction step including a 
resized picture generating step for generating a 
resized picture from the inputted picture data 
identified as still picture data, a frequency ana- 
lyzing step for executing a frequency analysis 
on said resized picture having been generated 
at said resized picture generating step, and a 
DC/lpartial AC component extracting step for 
taking out DC components and a part of AC 
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components obtained as a result of the analy- 
sis at said frequency analyzing step as a still 
picture feature data and a picture feature data 
of a picture within a picture data base where a 
still picture feature data having been generated 5 
at said still picture feature extraction step on 
the basis of a still picture data is stored 
together with said still picture data; 
a similar still picture retrieving step for retriev- 
ing the picture within said picture data base 10 
having a high correlation value with respect to 
the picture feature data generated on the basis 
of the inputted picture data as a candidate of 
similar still picture; 

a second picture feature data correlation meas- 15 
uring step for calculating a correlation value 
between a moving picture feature data of a 
moving picture in inputted moving picture data 
generated by a moving picture feature extrac- 
tion step including a feature data extracting 20 
frame selecting step for taking out some data 
corresponding to the entire frames or a portion 
of the frames from the picture data identified as 
moving picture data, selecting the data sub- 
jected to processing, a resized picture generat- 25 
ing step for generating a resized picture from 
data corresponding to each frame having been 
taken out, a frequency analyzing step for exe- 
cuting a frequency analysis on said resized pic- 
ture having been generated by said resized 30 
picture generating step, a DC/partial AC com- 
ponent extracting step for taking out DC com- 
ponents and a part of AC components obtained 
as a result of the analysis at said frequency 
analyzing step as a frame feature data, and a 35 
moving picture feature data producing step for 
generating a moving picture feature data by 
collecting said frame feature data outputted at 
said DC/Partial AC component extracting step, 
and a picture feature data of a picture within a 40 
picture data base where a moving picture fea- 
ture data having been generated at said mov- 
ing picture feature extraction step on the basis 
of a moving picture data is stored together with 
said moving picture data; and 45 
a similar moving picture retrieving step for 
retrieving a moving picture segment within said 
picture data base having a high correlation 
value with respect to the moving picture feature 
data generated on the basis of the inputted so 
moving picture data as a candidate of similar 
moving picture. 



moving picture feature extraction step on the basis 
of said inputted moving picture data, and a step of 
calculating in turn a partial correlation value 
between a part of the moving picture feature data of 
the moving picture included in said moving picture 
data base and a moving picture feature data of 
each division of the divided moving picture feature 
data, and 

said similar picture retrieving step eliminates 
the moving picture segment included in said mov- 
ing picture data base from the candidates of similar 
moving pictures when it has a feature data with a 
partial correlation value lower than a predetermined 
threshold value. 

31. A picture retrieving method as claimed in claim 27, 
wherein 

said second picture feature data correlation 
measuring step has a step of dividing the moving 
picture feature data having been generated by said 
moving picture feature extraction step on the basis 
of said inputted moving picture data, and a step of 
calculating in turn a partial correlation value 
between a part of the moving picture feature data of 
the moving picture included in said moving picture 
data base and a partial moving picture feature data 
of each division of the divided moving picture fea- 
ture data, and 

said similar picture retrieving step eliminates 
the moving picture segment included in said mov- 
ing picture data base from the candidates of similar 
moving pictures at the time when it is acknowl- 
edged that a number of partial moving picture fea- 
ture data having a feature data with a partial 
correlation value lower than a predetermined 
threshold value surpasses a predetermined thresh- 
old value. 

32. A picture retrieving method as claimed in claim 25, 
28, 29, 30, or 31 , wherein 

said resized picture generating step divides 
a picture into blocks, and takes out from each block 
color information representative of each block. 

33. A picture retrieving method as claimed in claim 25, 
28, 29, 30, or 31 , wherein 

said resized picture generating step takes 
out DC components from said each block when 
said picture data is compressed by blocks. 



30. A picture retrieving method as claimed in claim 27, 
wherein 55 

said second picture feature data correlation 
measuring step has a step of dividing the moving 
picture feature data having been generated by said 
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